Evaluation of Distance Measures Between Gaussian Mixture Models of MFCCs

In music similarity and in the related task of genre classification, a distance measure between Gaussian mixture models is frequently needed. We present a comparison of the KullbackLeibler distance, the earth movers distance and the normalized L2 distance for this application. Although the normalized L2 distance was slightly inferior to the Kullback-Leibler distance with respect to classification performance, it has the advantage of obeying the triangle inequality, which allows for efficient searching. 1. A Statistical Timbre Model We use the common approach of extracting mel-frequency cepstral coefficients (MFCCs) from a song, model them by a Gaussian mixture model (GMM) and use a distance measure between the GMMs as a measure of the musical distance between the songs [2, 4, 6]. 1.1 Mel-Frequency Cepstral Coefficients MFCCs are a compact, perceptually based representation of speech frames [3]. They are computed as follows: 1. Estimate the log-amplitude or log-power spectrum of 20– 30 ms of speech. 2. Sum the contents of neighboring frequency bins in overlapping bands distributed according to the mel-scale. 3. Compute the discrete cosine transform of the bands. 4. Discard high frequency coefficients from the cosine transform. 1.2 Gaussian Mixture Models We model the MFCCs from each song by a Gaussian mixture model (GMM): p(x)= K ∑ k=1 1 √ |2πΣk | exp ( −1 2(x−μk)TΣ−1 k (x−μk)), where K is the number of components. For K = 1, a closedform expression exists for the maximum-likelihood estimate of the parameters. For K > 1, the k-means algorithm and optionally the expectation-maximization algorithm are needed. 2. Distance Measures Between GMMs As distance measure between the GMMs, we have evaluated the symmetrized Kullback-Leibler distance, the earth movers distance and the normalized L2 distance. 2.1 Kullback-Leibler Distance The KL distance is given by dKL(p1, p2) = ∫ p1(x) log p1(x) p2(x)dx. (1) As the KL distance is not symmetric, we use a symmetrized version, dsKL(p1, p2) = dKL(p1, p2) + dKL(p2, p1). (2) For Gaussian mixtures, a closed form expression for dKL(p1, p2) only exists for K = 1. For K > 1, dKL(p1, p2) is estimated using stochastic integration or the approximation in [5]. 2.2 Earth Movers Distance The earth movers distance (EMD) is the minimum cost of changing one mixture into another when the cost of moving probability mass from component m in the first mixture to component n in the second mixture, cmn, is given [7, 4]. Let a1k be the weights of the Gaussians in p1(x), and a2k the weights of p2(x), then dEMD(p1, p2) is given by dEMD(p1, p2) = min∑ m ∑ n cmnfmn (3) subject to fij ≥ 0 (4) ∑